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DETERMINING CUSTOMER FEEDBACK 
BACKGROUND 

[001] Businesses and other concerns are continuing to look for better and more cost- 
effective ways of pCTforming maiketing campaigns as well as managing customer 
relationships. With unproved communications technology and the development of the 
World Wide Web, direct maiketing campaigns are often performed "on-line"; that is, 
mariceting is performed by communications ova: a data network, such as the Mtemet. 
Such communications are often in the form of electronic mail (e-mail) sent directly to 
potential customers. Alternatively, as users browse through various websites, 
advertisement banners are displayed on various parts of the web pages being viewed by 
theus^. 

[002] Improvements in technology have also made collecting customer feedback more 
convenient. For example, if a customer wishes to express a complaint or to provide some 
other type of feedback, the customer is able to send an e-mail directly to the customer 
service center of the respective business, m addition to or instead of calling the customer 
service center by telephone. Another possibiUty is that customers are able to access a 
feedback page provided by a website of a business, with customers being able to type in 
their feedback directly into a comments box in the displayed web page. 

[003] Traditionally, customer service rq>resentatives have to manually review the 
feedback provided by customers to determine whether customos are satisfied or not with 
goods or services provided by a business. For large businesses, the amount of feedback 
can be massive, and the amount of resources that are needed to review the feedback of 
customers or users can be quite large and costly. 

SUMMARY 

[004] In general, an improved method and apparatus is provided for determining 
customer feedback. For example, a method executable by a system comprises receiving 
customer feedback, analyzing words in the customer feedback, and associating at least 
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some of the words with respective one or more values. An indication to rate customer 
feedback is generated based on the one or more values* 

[005] Other or altemative features will become more apparent from the following 
description, from the claims, and from the drawings. 



RRTEF DESCRIPTION OF THE DRAWINGS 
[006] Fig. 1 is a block diagram of an example arrangement of systems, including a 
database system in accordance with an embodiment. 

[007] Fig. 2 is a block diagram of components of the database system of Fig. 1. 
[008] Fig. 3 illustrates tables stored in the database system of Fig. 1 . 
[009] Fig. 4 illustrates an array of words and associated raw scores. 
[0010] Fig. 5 is a flow diagram of a process in accordance with an embodiment. 



DETAILED DESCRIPTION 
^ [00 11] In the following description, numerous details are set forth to provide an 

2 understanding of the present invention. However, it will be understood by those skilled 

f4 hx the art that the present invention may be practiced without these details and that 

m 

numerous variations or modifications from the described embodiments are possible. 



[0012] Fig. 1 shows an arrangement of systems that are associated with an enterprise, 
such as a business, educational entity, government agency, or other concem. The 
enterprise mcludes various stations 12 to srad and/or receive electronic mail (e-mail). In 
addition, a web server 14 is provided that enables customers to access various web pages 
associated with the enterprise. Also, various telephones 16 are part of the enterprise. 
These telephones 16 are used by customer representatives, for example, to communicate 
with customers. As used here, a "customer** refers to any person that uses or purchases a 
good or service provided by an enterprise. 

[0013] The telephones 16 are coupled to one or more translator systems 18, each 
including a voice-to-text translator 20. Thus, any conversation between the customer 
representative (using a telephone 16) and a remote customer monitored by the translator 
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system 18 is converted by the voice-to-text translator 20 into text format. In one 
embodiment, the voice-to-text translator 20 is implemented as software. However, in 
other embodiments, the voice-to-text translator 20 is implemented as hardware or a 
combination of hardware and software. 

[0014] The e-mail stations 12, web server 14, telephones 16, and translator systems 18 
are part of the customer service infrastructure of the enterprise. The e-mail stations 12 
and web server 14 are coupled to a public network 22. One example of the public 
network 22 is the Internet. The e-mail stations 12 and web server 14 are also coupled to 
an intranet 24, which is a private network of the enterprise. Examples of the intranet 24 
include a local area network (LAN), a wide area network (WAN), a virtual private 
network (VPN), or any other network. 

j2 [001 5] A database system 26 is also coupled to the intranet 24. In the arrangement of 

Fig. 1, each of the e-mail stations 12, web server 14, and translator systems 18 are able to 
issue statements to the database system to write data to the database system 26 for 
storage. The database system 26 includes database software 28 that manages access and 
updates of data stored in a storage system 30. 

[0016] Another system that is coupled to the intranet 24 is a report client 36. The report 
client 36 is able to access various features of the database system 26 to determine 
customer feedback. In accordance with some embodunents of the invention, the report 
client uses data stored in the database system 26 according to user-defined data types 
(UDTs) as well as methods associated with the UDTs in the database system 26 to 
summarize or rate the customer feedback. 

[0017] In one example, the database system 26 is a relational database management 
system (RDBMS). An RDBMS stores data in relational tables that are made up of rows 
and coluirms. Each row represmts an occurrence of an entity defined by a table, with an 
entity being a person, place, or thing about which the table contains information. 

[0018] To extract data from, or to update, a relational table, queries according to a 
standard database query language are used (e.g.. Structured Query Language or SQL). 
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One version of SQL is provided by the SQL-92 Standard. Another version of SQL is 
provided by the SQL-99 Standard, also referred to as the SQL-3 Standard. 

[0019] One enhancement provided by SQL-99 over SQL-92 is the definition of user- 
defined data types (UDTs) in addition to predefined data types. Examples of predefined 
data types include the number data type for representing numeric values, character string 
data type for representing a sequence of characters, and so forth. A characteristic of 
predefined SQL data types is that the inherent structure of the data type is already defined 
by the SQL Standard. UDTs, on the otiier hand, are not predefined; instead, UDTs are 
created by programmers, such as those working for a database vendor, customer, and so 
forth. 

[0020] In accordance with some embodiments of the invention, table or tables stored in 
the storage system 30 of the database system 26 contain data according to a UDT 
(referred to as FeedbackUDT) to enable efficient and convenient detemunation of 
customer feedback based on information provided by the e-mail stations 12, web swver 
14, and/or translator systems 18. hi addition to the FeedbackUDT, feedback rating 
ioutine(s) 34 are also part of the database software 28. hi one embodiment, the feedback 
rating routine(s) 34 are UDT methods. One or more methods are associated with a UDT. 
Methods are routines or functions that are invoked to perform tasks with reject to UDT 
data. 

[0021] As shown in Fig. 2, according to one arrangement, the database system 26 
includes a plurality of nodes 1 16A, 1 16B, and 1 16C that are coupled together by an 
interconnect layer 115. The node 1 16A is coupled to the mtranet 24, and in the illustrated 
embodiment, includes a parsing engine (PE) or query coordmator 118. The parsing 
engine 118 interprets a query, checks the query for proper SQL syntax, and sends out 
executable actions to be performed by the nodes 1 16B, 1 16C. 

[0022] Each of the nodes 1 16B, 1 16C includes one or more access modules 122. One 
example of the access module 122 is the access module processor (AMP) used in some 
TERADATA® database systems fi:om NCR Corporation. The access modules 124 and 
parsing engine 1 18 are considered part of the database management software 28. The 
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access module 122 is responsible for managiiig access to respective portions of a 
database. As shown in Fig. 2, each access module 122 manages access to data stored in a 
respective storage module 124. Although shown as discrete components, the storage 
module 124 can be part of the same storage system (e.g., storage system 30 in Fig. 1), 
with the storage modules 124 representing different partitions of the storage system. 

[0023] Each storage module 124 stores one or more tables. Because the database system 
26 shown in Fig. 2 is a parallel database system that aiables concurrent access of 
different portions of a table, each table is distributed among plural storage modules 124. 
Note that each node 1 16 can contain one access mo<Me 122 or plural access modules 
122. 

O [0024] In a different embodimait, instead of having plural nodes, the parallel database 
p system 26 is implemented as a single node multiprocessing systan that has plural 
^ processors. In yet another embodiment, a uni-processor database system is used. Thus, 
il embodiments of the invention are applicable to either multiprocessing database systems 

w 

^ or non-multiprocessing database systems. 

^ [0025] As shown m Fig. 3, in accordance witii one embodiment, a feedback rating table 
P 200 is stored in the database system 26. In addition, a customer feedback table 202 is 
P also stored in the database system 26. In the database system arrangement shown in Fig. 
2, each of the tables 200 and 202 is distributed across plural access modules 122 and 
storage modules 124. In other words, the tables 200 and 202 are divided into multiple 
portions for storage on respective storage modules 124. Using tiie data stored in the 
tables 200 and 202, as well as the feedback rating n)utine(s) 34, an efficient and 
convenient mechanism is provided for determining customer feedback. 

[0026] In one unplementation, text contained in e-mails received by the e-mail stations 
12 are stored in the customer feedback table 202. Also, the text of any feedback received 
at the web server 14 is also stored in the customer feedback table 202. Conversational 
feedback received by the translator systems 18 are tianslated to text, which are written to 
the customer feedback table 202. The e-mail stations 12, web server 14, and translator 
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systems 18 are capable of generating SQL queries, such as INSERT statements, to add 
rows to the customs feedback table 202. 
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[0027] Using the feedback data stored in the customer feedback table 202, and using a 
rating scheme stored in the feedback rating table 200, a "score" can be derived &om the 
customer feedback data. The "score" is an indication of a level of customer feedback, 
such as customer satisfaction or dissatisfaction, customer anotion, customer ^pro^ or 
disj^roval, and so forth. 

[0028] As shown in Fig. 3, the customer feedback table 202 includes various columns, 
including a CUSTOMER_ID column (to provide unique identifiers for respective 
customers or users), a feedback column (which contains the feedback data from the 
customer), and a Timestamp column (which contains an indication of flie date and time of 
flie collected feedback). The Feedback column is a string of text the containing customer 
feedback. 

[0029] The feedback rating table 200 includes an ID column and a Feedback_Score 
column. Each entry of the Feedback_Score column contains an array of keywords and 
associated scores. In accordance with one embodunait, the arrays stored in the 
Feedback_Score column are according to the FeedbackUDT. 

[0030] An example of an array stored in die Feedback_Score column is shown in Fig. 4. 
In the Fig. 4 array, the various words that are associated wilh respective scores are 
"anger," "joy," "sad," "surprise," and "disgust." The words shown are provided as an 
example only, and are not meant to be exhaustive. Many other words can be associated 
with raw scores, with such other words indicating user satisfaction or dissatisfaction, 
approval or disapproval, or any other type of feedback lhat an enterprise is interested in. 

[0031] hi the example of Fig. 4, the word "anger" has a raw score of -5, which is a 
negative number to indicate a negative customer feedback. On the other hand, the word 
"joy** has a raw score of +7, which is a positive number to indicate a positive customer 
feedback. The word "sad" has a raw score of -2, the word "disgust" has a raw score of 
-4, and the word "surprise" has a raw score of +7. 
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[0032] As shown in Fig. 4, the words can also be associated with adjectives, such as 
"very" or "extremely." Such adjectives increase the score of the word. For example, the 
word "very" adds a factor of 2 to the score. Thus, the terms "very sad" would have a 
score of 2*-2=-4. The word "extremely" adds a factor of 4 to the score. Thus, the terms 
"«ctremely joyful" would have a score of 4*7=28. 

[0033] Each row of the feedback ratmg table 200 is associated with a unique ID. For 
example, in Fig. 3, the firat row has IDl, while the second row has E)2. The different 
rows contain differrait arrays ofkeywords and associated scores. This provides flexibility 
for an enterprise in rating customer feedback. For example, different arrays can be used 
for persons in different age groups, with difierent backgrounds, in different professions, 

I* 

Q and so forth. 

O 

VI [0034] As shown in Fig. 2, two UDT methods are associated with the FeedbackUDT. 
d The methods include a Load_FeedbackScore method 140 and a Grade_Feedback method 
142. In one example embodiment, the Load_FeedbackScore method 140 has the 

^ following syntax: Load^FeedbackScore (Feedback_Score: FeedbackUDT, 

Q FeedbackJTerm: string, Raw_Score: integw). 

il . 

^ r00351 The Load FeedbackScore method 140 loads a word (Feedback_Term) and 
i& associated score (Raw_Score) into a Feedback_Score object in the feedback ratmg table 
200. Although not shown, the corresponding ID oftheFeedback_Score object to which 
the word and score are to be added can also be specified as a parameter in the 
Load_FeedbackScore method 140. Alternatively, instead of loading one term at a time, 
an entire array of words and scores can be specified in the Feedback_Tain and 
Raw_Score parameters for loading into the feedback rating table 200. 

[0036] Additional parameters are also specified in the Load_FeedbackScore method 140 
to add values for the score modifiers "V ery," '^Extremely," and so forth. 

[0037] The Grade_Feedback routine 142 has the following syntax: Score = 
Grade_Feedback (Feedback_Score: FeedbackUDT, Feedback_List: array of string, 
Textdoc: Text). 
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[0038] The Grade^Feedback method 142 returns a score that indicates the intensity 
(either positive or negative) of the customer feedback contained in Textdoc. The 
parameter Textdoc contains a string including the text of the customer feedback, such as 
text in an e-mail, text received at the web server 14, and/or text provided by a translator 
system 18. 

[0039] The parameter Feedback_List contains either a null value or a list of words that 
are of interest. If Feedback_List has a null value, then all words m Textdoc are examined 
for matching to the UDT array Feedback^Score in the feedback rating table 20. 
However, if Feedback_List has a list of one or more words, then only those words in 
Textdoc matching the list are considered, with a score derived for those words by 
P accessing Feed.back_Score in the feedback rating table 200. 

m [0040] An example statement for perfoiming a join of the customer feedback table 202 

IP 

m and feedback rating table 200 to rate a customer feedback is provided below. 



[0041] SELECT Grade_Feedback (Feedback_Score, Feedback_List, Textdoc) 

FROM CUSTOMER_FEEDBACK_TABLE, FEEDBACK_RATING_TABLE 
WHERE Grade_Feedback (Feedback_Score, Feedback_List, Textdoc) > 10 AND 
FEEDBACK_RATING_TABLE.ID=ID1 

[0042] As specified in the statement above, the Feedback_Score array having identifier 
IDl in the feedback rating table 200 is used to calculate a score for the text in Textdoc 
that match words in Feedback_List (assuming a non-null value). If GradeJFeedback ( ) 
returns a value greater than 10, thai the value of Grade_Feedback ( ) is selected for 
ou^ut in the query result 

[0043] A process according to one embodiment of the invention is shown in Fig. 5. A 
system, such as the report client 36 or another system, creates (at 300) the customer 
feedback table 202 and the feedback rating table 200 in tiie database system 26. Next, the 
system loads (at 301) values into the feedback rating table 200 by calling the 
Load_FeedbackScore method 140. The Lx)ad_FeedbackScore method 140 can be 
invoked a plurality of times to load multiple values, words, or entries. 
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[0044] The one or more client systems, including the e-mail systems 12, web server 14, 
and translator systems 18, receive feedback from customers (at 302). The feedback is 
provided in some format, such as text foraiat, in queries submitted (at 304) to the 
database system 26 for insertion into the customer feedback table 202. 

[0045] The report client 36 next determines if it has received a request to determine the 
feedback score (at 306). The request can be submitted by a user or an application. Li 
response, the report client 36 builds (at 308) a query, which can be sunilar to the 
SELECT statement shown above. The query is then submitted to a database system 26 to 
calculate one or more feedback scores (at 310). Upon receipt of flie response from the 
database system 26, the report client 36 outputs (at 3 12) the one or more scores. As 
examples, the output 36 is displayed, printed, or presented by some other means. 

[0046] Instructions of the various software routines or modules discussed herein (such as 
the methods 140 and 142, flie access modules 122) are stored on one or more storage 
modules in the corresponding systems and loaded for execution on corresponding control 
units or processors. The control units or processors include microprocessors, 
microcontrollers, processor modules or subsystems (including one or more 
microprocessors or microcontrollers), or other control or computing devices. As used 
here, a "controller" refers to hardware, software, or a combination thereof. A 
"controller" can refer to a single component or to plural components (whether software or 
hardware). 

[0047] Data and instructions (of the various software routines or modules) are stored in 
respective storage modules, which are implemented as one or more machine-readable 
storage media. The storage media include different forms of memory including 
semiconductor memory devices such as dynamic or static random access memories 
(DRAMs or SRAMs), erasable and prograrmnable read-only memories (EPROMs), 
electrically erasable and programmable read-only memories (EEPROMs) and flash 
memories; magnetic disks such as fixed, floppy and removable disks; other magnetic 
media including t^e; and optical media such as compact disks (CDs) or digital video 
disks (DVDs). 



[0048] The instructions of the software routines or modules are loaded or transported to 
each device or system in one of many different ways. For example, code segments 
including instructions stored on floppy disks, CD or DVD media, a hard disk, or 
transported through a network interface card, modem, or other interface device are loaded 
into the device or system and executed as corresponding software modules or layers* In 
tihe loading or transport process, data signals that are embodied in carrier waves 
(transmitted over telephone lines, network lines, wireless links, cables, and the like) 
communicate the code segments, including instructions, to the device or system. Such 
carrier waves are m the form of electrical, optical, acoustical, electromi^etic, or other 
types of signals. 

[0049] While the invention has been disclosed with respect to a limited number of 
embodiments, those skilled in the art will appreciate numerous modifications and 
variations there from. It is intended that the appended claims cover such modifications 
and variations as fall within the true spirit and scope of the inventioiL 



10 



